(19) 



3 



Europaisches Potentamt 
European Patent Office 
Office europeen des brevets 



(12) 



(43) Date of publication: 

02.05.2002 Bulletin 2002/18 

(21) Application number: 02000566.6 

(22) Date of filing: 27.09.1996 



(H) EP1 202 568 A2 

EUROPEAN PATENT APPLICATION 

(51) int ci 7: H 04 N 5/92 



(84) Designated Contracting States: 


• Tsuga, Kazuhlro 


DE FR GB 


Takarazuka-shl, Hyogo 665 (JP) 


Designated Extension States: 


• Hamasaka, Hiroshi 


AL LT LV SI 


Hlrakata-shl, Osaka 573 (JP) 




• Ishlhara, Hideshl 


(30) Priority: 29.09.1995 JP 25273595 


Katano-shl, Osaka 576 (JP) 


28.02.1996 JP 4158196 


• Nakamura, Kazuhiko 




Hirakata-shi, Osaka 573 (JP) 


(62) Document number(s) of the earlier application (s) in 


• Hasebe, Takumi 


accordance with Art. 76 EPC: 


Yawata-shl, Kyoto 614 (JP) 


99104408.2/0 930 783 




96932018.3/0 847196 


(74) Representative: ElsenfOhr, Speiser & Partner 




Marti nistrasse 24 


(71) Applicant: Matsushita Electric Industrial Co., Ltd. 


28195 Bremen (DE) 


Kadoma-shl, Osaka-fu 571-8501 (JP) 






Remarks: 


(72) Inventors: 


This application was filed on 10 - 01 - 2002 as a 


• Okada, Tomoyuki 


divisional application to the application mentioned 


Katano-shl, Osaka 576 (JP) 


under INID code 62. 


• Mori, Yoshihiro 




Hirakata-shi, Osaka 573 (JP) 





(54) Method and apparatus for encoding bitstreams for seamless reproduction 



(57) An optical disk having such a data structure that 
moving image data and audio data are naturally repro- 
duced under one title without stoppage (freeze), etc., of 
video display at the connections of system streams 
(VOB) in which the data are interleaved when the data 
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are reproduced by connecting the system streams 
(VOB) to each other. At least the first audio frame (Af) 
contains the same audio data in a plurality of branched 
stream systems (VOB) and at least the last GOP con- 
tains the same moving picture in a plurality of system 
streams (VOB) before connected. 
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efficiently utilize optical disk capacity, multi-scene con- 
trot whereby scene data common to plural titles and the 
desired scenes on the same time-base from within multi- 
scene periods containing plural scenes unique to par- 
ticular reproduction paths can be freely selected and re- s 
produced is desirable. 

[0014] However, when plural scenes unique to a re- 
production path within the multi-scene period are ar- 
ranged on the same time-base, the scene data must be 
contiguous. Unselected multi-scene data is therefore ro 
unavoidably inserted between the selected common 
scene data and the selected multi-scene data. The prob- 
lem this creates when reproducing multi-scene data is 
that reproduction is interrupted by this unselected scene 
data. - 15 

[001 5] When one of the multiple scenes is connected 
to common scene data, the difference between the vid- 
eo reproduction time and the audio reproduction time 
differs on each of the reproduction paths because of the 
offset between the audio and video frame reproduction 20 
times. As a result, the audio or video buffer underflows 
at the scene connection, causing video reproduction to 
stop ("freeze") or audio reproduction to stop ("mute"), 
and thus preventing seamless reproduction. It will also 
be obvious that the difference between the audio and 25 
video reproduction times can cause a buffer underflow 
state even when common scene data is connected 1:1. 
[0016] Therefore, the object of the present invention 
is to provide a data structure whereby multi-scene data 
can be naturally reproduced as a single title without the 30 
video presentation stopping ("freezing") at one-to-one, 
one-to-many, or many-to-many scene connections in 
the system stream; a method for generating a system 
stream having said data structure; a recording appara- 
tus and a reproduction apparatus for recording and re- 35 
producing said system stream; and a medium to which 
said system stream can be recorded and from which 
said system stream can be reproduced by said record- 
ing apparatus and reproduction apparatus. 
[0017] The present application is based upon Japa- *o 
nese Patent Application No. 7-252735 and 8-041581, 
which were filed on September 29, 1995 and February 
28, 1996, respectively, the entire contents of which are 
expressly incorporated by reference herein. 

45 

Disclosure of Invention 

[0018] The present invention has been developed 
with a view to substantially solving the above described 
disadvantages and has for its essential object to provide so 
an optical disk for recording more than one system 
stream containing audio data and video data, wherein 
the audio data and video data of the plural system 
streams recorded to the optical disk are interleaved 
such that the difference between the input start times of 55 
the video data and audio data to the video buffer in the 
video decoder and the audio buffer in the audio decoder 
is less than the reproduction time of the number of audio 
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frames that can be stored in the audio buffer plus one 
audio frame. 

Brief Description of Drawings 
[0019] 

Fig. 1 is a graph schematically showing a structure 
of multi media bit stream according to the present 
invention, 

Fig. 2 is a block diagram showing an authoring en- j 
coder according to the present invention, | 
Fig. 3 is a block diagram showing an authoring de- } 
coder according to the present invention, ] 
Fig. 4 is a side view of an optical disk storing the j 
multi media bit stream of Fig. 1 , 
Fig. 5 is an enlarged view showing a portion con- 
fined by a circle of Fig. 4, 

Fig. 6 is an enlarged view showing a portion con- 
fined by a circle of Fig. 5, 

Fig. 7 is a side view showing a variation of the op- ; 
tical disk of Fig. 4, 

Fig. 8 is a side view showing another variation of j 
the optical disk of Fig. 4, 

Fig. 9 is a plan view showing one example of track 
path formed on the recording surface of the optical 
disk of Fig. 4, 

Fig. 10 is a plan view showing another example of 
track path formed on the recording surface of the 
optical disk of Fig. 4, 

Fig. 1 1 is a diagonal view schematically showing 
one example of a track path pattern formed on the 
optical disk of Fig. 7, 

Fig. 12 is a plan view showing another example of 
track path formed on the recording surface of the 
optical disk of Fig. 7, 

Fig. 13 is a diagonal view schematically showing 
one example of a track path pattern formed on the 
optical disk of Fig. 8, 

Fig. 14 is a plan view showing another example of 
track path formed on the recording surface of the 
optical disk of Fig. 8, 

Fig. 1 5 is a flow chart showing details of the decoder 
synchronization process of Fig. 66, 
Fig. 16 is a graph schematically showing the struc- 
ture of multimedia bit stream for use in Digital Video 
Disk system according to the present invention, 
Fig. 1 7 is a graph schematically showing the encod- 
ed video stream according to the present invention, 
Fig. 1 8 is a graph schematically showing an internal 
structure of a video zone of Fig. 16. 
Fig. 1 9 is a graph schematically showing the stream 
management information according to the present 
invention, 

Fig. 20 is a graph schematically showing the struc- 
ture the navigation pack NV of Fig. 1 7, 
Fig. 21 is a graph in assistance of explaining a con- 
cept of parental lock playback control according to 
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the present invention, 

Fig. 22 is a graph schematically showing the data 
structure used in a digital video disk system accord- 
ing to the present invention, 

Fig. 23 is a graph in assistance of explaining a con- 5 
cept of Multi-angle scene control according to the 
present invention, 

Fig. 24 is a graph in assistance of explaining a con- 
cept of multi scene data connection, 
Fig. 25 Is a block diagram showing a DVD encoder 10 
according to the present invention, 
Fig. 26 is a block diagram showing a DVD decoder 
according to the present invention, 
Fig. 27 is a graph schematically showing an encod- 
ing information table generated by the encoding 15 
system controller of Fig. 25, 
Fig. 28 is a graph schematically showing an encod- 
ing information tables, 

Fig. 29 is a graph schematically showing an encod- 
ing parameters used by the video encoder of Fig. 20 
25, 

Fig. 30 is a graph schematically showing an exam- 
ple of the contents of the program chain information 
according to the present invention, 
Fig. 31 is a graph schematically showing another 25 
example of the contents of the program chain infor- 
mation according to the present invention, 
Fig. 32 is a flow chart showing the encode param- 
eters generating operation for a system stream con- 
taining a single scene, 30 
Fig. 33 is a graph in assistance of explaining a con- 
cept of multi-angle scene control according to the 
present in invention, 

Fig. 34 is a flow chart, formed by Figs. 34A and 34B, 
showing an operation of the DVD encoder of Fig. 35 
25, 

Fig. 35 is a flow chart showing detailed of the en- 
code parameter production sub-routine of Fig. 34, 
Fig. 36 is a flow chart showing the detailed of the 
VOB data setting routine of Fig. 35, *o 
Fig. 37 is a flow chart showing the encode param- 
eters generating operation for a seamless switch- 
ing, 

Fig. 38 is a flow chart showing the encode param- 
eters generating operation for a system stream, <5 
Fig. 39 is a graph showing simulated results of data 
input/output to the video buffer and audio buffer of 
the DVD decoder of Fig. 26, 
Fig. 40 is a graph in assistance of explaining a con- 
cept of parental control according to the present in- 50 
vention, 

Fig. 41 is a graph in assistance of explaining the 
data input/output to the video buffer of the DVD de- 
coder DCD shown in Fig. 26 during contiguous re- 
production, 55 
Fig. 42 is a graph in assistance of explaining a pos- 
sible problem under a parental lock control example 
shown in Fig. 40, 



Fig. 43 is a graph in assistance of explaining a re- 
production gap generated under parental lock con- 
trol, 

Fig. 44 is a graph showing a system streams pro- 
duced according to the present invention, 
Fig. 45 is a graph in assistance of explaining an op- 
eration whereby these system streams are connect- 
ed, 

Fig. 46 is a graph in assistance of explaining a meth- 
od of generating a system streams, 
Fig. 47 is a graph in assistance of explaining anoth- 
er method of producing a system stream, 
Fig. 48 is a graph showing a structure of the end of 
the second common system stream and the begin- 
nings of the two parental lock control system 
streams, 

Fig. 49 is a graph in assistance of explaing the dif- 
ference in the video reproduction time and audio re- 
production time of different reproduction paths, 
Fig. 50 is a block diagram showing an internal struc- 
ture of the system encoder in the DVD encoder of 
Fig. 25, 

Fig. 51 is a graph showing a structure of the end of 
the two parental lock control system streams and 
the beginning of the following common system 
stream Sse, 

Fig. 52 is a graph in assistance of explaining the 
difference in the video reproduction time and audio 
reproduction time of different reproduction paths, 
Fig. 53 is a flow chart showing details of system 
stream producing routine of Fig. 34, 
Fig. 54 is a graph in assistance of explaining an op- 
eration to calculate an audio data movement 
MFApl, 

Fig. 55 is a graph in assistance of explaining an op- 
eration to calculate an audio data movement 
MFAp2, 

Fig. 56 is a block diagram showing an internal struc- 
ture of the synchronizer of Fig. 26, 
Fig. 57 is a flow chart showing an operation execut- 
ed by the audio decoder controller of Fig. 26, 
Figs. 58 and 59 are graphs showing decoding infor- 
mation table produced by the decoding system con- 
troller of Fig. 26, 

Fig. 60 is a flow chart showing the operation of the 
DVD decoder DCD of Fig. 26, 
Fig. 61 is a flow chart showing details of reproduc- 
tion extracted PGC routing of Fig. 60, 
Fig. 62 is a flow chat showing details of the stream 
buffer data transfer process according to the 
present invention, 

Fig. 63 is a flow chart showing details of the non 
multi-angle decoding process of Fig. 62, 
Fig. 64 is a flow chart showing details of the non- 
multi-angled interleave process of Fig. 63, 
Fig. 65 is a flow chart showing details of the non- 
multi-angled contiguous block process, 
Fig. 66 is a flow chart showing details of decoding 
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data process of Fig. 64, performed by the stream 
buffer, is shown, 

Fig. 67 is a graph schematically showing an actual 
arrangement of data blocks recorded to a data re- 
cording track on a recording medium according to 
the present invention, 

Fig. 68 is a graph schematically showing contigu- 
ous block regions and interleaved block regions ar- 
ray, 

Fig. 69 is a graph schematically showing a content 
of a VTS title VOBS according to the present inven- 
tion, and 

Fig. 70 is a graph schematically showing an internal 
data structure of the interleaved block regions ac- 
cording to the present invention. 

Best Mode for Carrying Out the Invention 

[0020] The prevent invention is detailedly described 
with reference to the accompanying drawings. 

Data structure of the authoring system 

[0021] The logic structure of the multimedia data bit- 
stream processed using the recording apparatus, re- 
cording medium, reproduction apparatus, and authoring 
system according to the present invention is described 
first below with reference to Fig. 1 . 
[0022] In this structure, one title refers to the combi- 
nation of video and audio data expressing program con- 
tent recognized by a user for education, entertainment, 
or other purpose. Referenced to a motion picture (mov- 
ie), one title may correspond to the content of an entire 
movie, or to just one scene within said movie. 
[0023] A video title set (VTS) comprises the bitstream 
data containing the information for a specific number of 
titles. More specifically, each VTS comprises the video, 
audio, and other reproduction data representing the 
content of each title in the set, and control data for con- 
trolling the content data. 

[0024] The video zone VZ is the video data unit proc- 
essed by the authoring system, and comprises a spe- 
cific number of video title sets. More spec if teal fy, each 
video zone is a linear sequence of K + 1 video title sets 
numbered VTS #0 - VTS #K where K is an integer value 
of zero or greater. One video title set, preferably the first 
video title set VTS #0, is used as the video manager 
describing the content information of the titles contained 
in each video title set. 

[0025] The multimedia bitstream MBS is the largest 
control unit of the multimedia data bitstream handled by 
the authoring system of the present invention, and com- 
prises plural video zones vz. 

Authoring encoder EC 

[0026] A preferred embodiment of the authoring en- 
coder EC according to the present invention for gener- 



ating a new multimedia bitstream MBS by re-encoding 
the original multimedia bitstream MBS according to the 
scenario desired by the user is shown in Fig. 2. Note 
that the original multimedia bitstream MBS comprises a 

s video stream St1 containing the video information, a 
sub-picture stream St3 containing caption text and other 
auxiliary video information, and the audio stream St5 
containing the audio information. 
[0027] The video and audio streams are the bit- 

io streams containing the video and audio information ob- 
tained from the source within a particular period of time. 
The sub-picture stream is a bitstream containing mo- 
mentary video information relevant to a particular scene. 
The sub-picture data encoded to a single scene may be 

m captured to video memory and displayed continuously 
from the video memory for plural scenes as may be nec- 
essary. 

[0028] When this multimedia source data St1, St3, 
and St5 is obtained from a live broadcast, the video and 

20 audio signals are supplied in real-time from a video cam- 
era or other imaging source; when the multimedia 
source data is reproduced from a video tape or other 
recording medium, the audio and video signals are not 
real-time signals. 

25 [0029] While the multimedia source stream is shown 
in Fig. 2 as comprising these three source signals, this 
is for convenience only, and it should be noted that the 
multimedia source stream may contain more than three 
types of source signals, and may contain source data 

30 for different titles. Multimedia source data with audio, 
video, and sub-picture data for plural titles are referred 
to below as multi-title streams. 
[0030] As shown in Fig. 2, the authoring encoder EC 
comprises a scenario editor 1 00, encoding system con- 

35 troller 200, video encoder 300, video stream buffer 400, 
sub-picture encoder 500, sub-picture stream buffer 600, 
audio encoder 700, audio stream buffer 800, system en- 
coder 900, video zone formatter 1300, recorder 1200, 
and recording medium M. 

40 [0031 ] The video zone formatter 1 300 comprises vid- 
eo object (VOB) buffer 1000, formatter 1100, and vol- 
ume and file structure formatter 1 400. 
[0032] The bitstream encoded by the authoring en- 
coder EC of the present embodiment is recorded by way 

45 of example only to an optical disk. 

[0033] The scenario editor 1 00 of the authoring en- 
coder EC outputs the scenario data, i.e., the user-de- 
fined editing instructions. The scenario data controls ed- 
iting the corresponding parts of the multimedia bitstream 

so MBS according to the user's manipulation of the video, 
sub-picture, and audio components of the original mul- 
timedia title. This scenario editor 100 preferably com- 
prises a display, speakers), keyboard, CPU, and 
source stream buffer. The scenario editor 1 00 is con- 

55 nected to an external multimedia bitstream source from 
which the multimedia source data St1 , St3, and St5 are 
supplied. 

[0034] The user is thus able to reproduce the video 
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and audio components of the multimedia source data 
using the display and speaker to confirm the content of 
the generated title. The user is then able to edit the title 
content according to the desired scenario using the key- 
board, mouse, and other command input devices while 
confirming the content of the title on the display and 
speakers. The result of this multimedia data manipula- 
tion is the scenario data St7. 

[0035] The scenario data St 7 is basically a set of in- 
structions describing what source data is selected from 
all or a subset of the source data containing plural titles 
within a defined time period, and how the selected 
source data is reassembled to reproduce the scenario 
(sequence) intended by the user. Based on the instruc- 
tions received through the keyboard or other control de- 
vice, the CPU codes the position, length, and the relative 
time-based positions of the edited parts of the respec- 
tive multimedia source data streams St1 , St3, and St5 
to generate the scenario data St7. 
[0036] The source stream buffer has a specific capac- 
ity, and is used to delay the multimedia source data 
streams St1 , St3, and St5 a known time Td and then 
output streams St1, St3, and St5. 
[0037] This delay is required for synchronization with 
the editor encoding process. More specifically, when da- 
ta encoding and user generation of scenario data St7 
are executed simultaneously, i.e., when encoding im- 
mediately follows editing, time Td is required to deter- 
mine the content of the multimedia source data editing 
process based on the scenario data St7 as will be de- 
scribed further below. As a result, the multimedia source 
data must be delayed by time Td to synchronize the ed- 
iting process during the actual encoding operation. Be- 
cause this delay time Td is limited to the time required 
to synchronize the operation of the various system com- 
ponents in the case of sequential editing as described 
above, the source stream buffer is normally achieved by 
means of a high speed storage medium such as semi- 
conductor memory. 

[0038] During batch editing in which all multimedia 
source data is encoded at once ("batch encoded") after 
scenario data St7 is generated for the complete title, de- 
lay time Td must be long enough to process the com- 
plete title or longer. In this case, the source stream buffer 
may be a low speed, high capacity storage medium such 
as video tape, magnetic disk, or optical disk. 
[0039] The structure (type) of media used for the 
source stream buffer may therefore be determined ac- 
cording to the delay time Td required and the allowable 
manufacturing cost. 

[0040] The encoding system controller 200 is con- 
nected to the scenario editor 100 and receives the sce- 
nario data St7 therefrom. Based on the time-base posi- 
tion and length information of the edit segment con- 
tained in the scenario data St7, the encoding system 
controller 200 generates the encoding parameter sig- 
nals St9, St11 , and St13 for encoding the edit segment 
of the multimedia source data. The encoding signals 



St9, St 11 , and St1 3 supply the parameters used for vid- 
eo, sub-picture, and audio encoding, including the en- 
coding start and end timing. Note that multimedia source 
data St1 , St3, and St5 are output after delay time Td by 
5 the source stream buffer, and are therefore synchro- 
nized to encoding parameter signals St9, St11, and 
St13. 

[0041] More specifically, encoding parameter signal 
St9 is the video encoding signal specifying the encoding 

»o timing of video stream St1 to extract the encoding seg- 
ment from the video stream St1 and generate the video 
encoding unit. Encoding parameter signal St 11 is like- 
wise the sub-picture stream encoding signal used to 
generate the sub-picture encoding unit by specifying the 

15 encoding timing for sub-picture stream St3. Encoding 
parameter signal St 1 3 is the audio encoding signal used 
to generate the audio encoding unit by specifying the 
encoding timing for audio stream St5. 
[0042] Based on the time-base relationship between 

20 the encoding segments of streams St1 , St3, and St5 in 
the multimedia source data contained in scenario data 
St 7, the encoding system controller 200 generates the 
timing signals St21 , St23, and St25 arranging the en- 
coded multimedia-encoded stream in the specified time- 
rs base relationship. 

[0043] The encoding system controller 200 also gen- 
erates the reproduction time information IT defining the 
reproduction time of the title editing unit (video object, 
VOB), and the stream encoding data St33 defining the 

30 system encode parameters for multiplexing the encod- 
ed multimedia stream containing video, audio, and sub- 
picture data. Note that the reproduction time information 
IT and stream encoding data St33 are generated for the 
video object VOB of each title in one video zone VZ. 

35 [0044] The encoding system controller 200 also gen- 
erates the title sequence control signal St39, which de- 
clares the formatting parameters for formatting the title 
editing units VOB of each of the streams In a particular 
time-base relationship as a multimedia bitstream. More 

40 specifically, the title sequence control signal St39 is 
used to control the connections between the title editing 
units (VOB) of each title in the multimedia bitstream 
MBS, or to control the sequence of the Interleaved title 
editing unit (VOBs) interleaving the title editing units 

45 VOB of plural reproduction paths. 

[0045] The video encoder 300 is connected to the 
source stream buffer of the scenario editor 100 and to 
the encoding system controller 200, and receives there- 
from the video stream St1 and video encoding parame- 

50 ter signal St9, respectively. Encoding parameters sup- 
plied by the video encoding signal St9 include the en- 
coding start and end timing, bit rate, the encoding con- 
ditions for the encoding start and end, and the material 
type. Possible material types include NTSC or PAL vid- 

55 eo signal, and telecine converted material. Based on the 
video encoding parameter signal St9, the video encoder 
300 encodes a specific part of the video stream St1 to 
generate the encoded video stream St15. 
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[0046] The sub-picture encoder 500 is similarly con- 
nected to the source stream buffer of the scenario editor 
1 00 and to the encoding system controller 200, and re- 
ceives therefrom the sub-picture stream St3 and sub- 
picture encoding parameter signal St11, respectively. 
Based on the sub-picture encoding parameter signal 
St11, the sub-picture encoder 500 encodes a specific 
part of the sub-picture stream St3 to generate the en- 
coded sub-picture stream St17. 
[0047] The audio encoder 700 is also connected to 
the source stream buffer of the scenario editor 100 and 
to the encoding system controller 200, and receives 
therefrom the audio stream St5 and audio encoding pa- 
rameter signal St13, which supplies the encoding start 
and end timing. Based on the audio encoding parameter 
signal St13, the audio encoder 700 encodes a specific 
part of the audio stream St5 to generate the encoded 
audio stream St 19. 

[0048] The video stream buffer 400 is connected to 
the video encoder 300 and to the encoding system con- 
troller 200. The video stream buffer 400 stores the en- 
coded video stream St1 5 input from the video encoder 
300, and outputs the stored encoded video stream St1 5 
as the time-delayed encoded video stream St27 based 
on the timing signal St21 supplied from the encoding 
system controller 200. 

[0049] The sub-picture stream buffer 600 is similarly 
connected to the sub-picture encoder 500 and to the en- 
coding system controller 200. The sub-picture stream 
buffer 600 stores the encoded sub-picture stream St17 
output from the sub-picture encoder 500, and then out- 
puts the stored encoded sub-picture stream St17 as 
time-delayed encoded sub-picture stream St29 based 
on the timing signal St23 supplied from the encoding 
system controller 200. 

[0050] The audio stream buffer BOO is similarly con- 
nected to the audio encoder 700 and to the encoding 
system controller 200. The audio stream buffer 800 
stores the encoded audio stream St1 9 input from the 
audio encoder 700, and then outputs the encoded audio 
stream St1 9 as the time-delayed encoded audio stream 
St31 based on the timing signal St25 supplied from the 
encoding system controller 200. 
[0051] The system encoder 900 is connected to the 
video stream buffer 400, sub-picture stream buffer 600, 
audio stream buffer 800, and the encoding system con- 
troller 200, and is respectively supplied thereby with the 
time-delayed encoded video stream St27, time-delayed 
encoded sub-picture stream St29, time-delayed encod- 
ed audio stream St31 , and the stream encoding data 
St33. Note that the system encoder 900 is a multiplexer 
that multiplexes the time-delayed streams St27, St29, 
and St3l based on the stream encoding data St33 (tim- 
ing signal) to generate title editing unit (VOB) St35. The 
stream encoding data St33 contains the system encod- 
ing parameters, including the encoding start and end 
timing. 

[0052] The video zone formatter 1 300 is connected to 



the system encoder 900 and the encoding system con- 
troller 200 from which the title editing unit (VOB) St35 
and title sequence control signal St39 (timing signal) are 
respectively supplied. The title sequence control signal 

5 St 3 9 contains the formatting start and end timing, and 
the formatting parameters used to generate (format) a 
multimedia bitstream MBS. The video zone formatter 
1300 rearranges the title editing units (VOB) St35 in one 
video zone VZ in the scenario sequence defined by the 

10 user based on the title sequence control signal St39 to 
generate the edited multimedia stream data St43. 
[0053] The multimedia bitstream MBS St43 edited ac- 
cording to the user-defined scenario is then sent to the 
recorder 1 200. The recorder 1 200 processes the edited 

15 multimedia stream data St43 to the data stream St45 
format of the recording medium M, and thus records the 
formatted data stream St45 to the recording medium M. 
Note that the multimedia bitstream MBS recorded to the 
recording medium M contains the volume file structure 

20 VFS, which includes the physical address of the data on 
the recording medium generated by the video zone for- 
matter 1300. 

[0054] Note that the encoded multimedia bitstream 
MBS St 35 may be output directly to the decoder to im- 
25 mediately reproduce the edited title content. It will be 
obvious that the output multimedia bitstream MBS will 
not in this case contain the volume file structure VFS. 

Authoring decoder DC 

30 

[0055] A preferred embodiment of the authoring de- 
coder DC used to decode the multimedia bitstream MBS 
edited by the authoring encoder EC of the present in- 
vention, and thereby reproduce the content of each title 

35 unit according to the user-defined scenario, is described 
next below with reference to Fig. 3. Note that in the pre- 
ferred embodiment described below the multimedia bit- 
stream St45 encoded by the authoring encoder EC is 
recorded to the recording medium M. 

^0 [0056] As shown in Fig. 3, the authoring decoder DC 
comprises a multimedia bitstream producer 2000, sce- 
nario selector 2100, decoding system controller 2300, 
stream buffer 2400, system decoder 2500, video buffer 
2600, sub-picture buffer 2700, audio buffer 2800, syn- 

45 chronizer 2900, video decoder 3800, sub-picture decod- 
er 3100, audio decoder 3200, synthesizer 3500, video 
data output terminal 3600, and audio data output termi- 
nal 3700. 

[0057] The bitstream producer 2000 comprises a re- 
50 cording media drive unit 2004 for driving the recording 
medium M; a reading head 2006 for reading the infor- 
mation recorded to the recording medium M and pro- 
ducing the binary read signal St57; a signal processor 
2008 for variously processing the read signal St57 to 
55 generate the reproduced bitstream St61 ; and a repro- 
duction controller 2002. 

[0058] The reproduction controller 2002 is connected 
to the decoding system controller 2300 from which the 
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multimedia bitstream reproduction control signal St53 is 
supplied, and in turn generates the reproduction control 
signals St55 and St59 respectively controlling the re- 
cording media drive unit (motor) 2004 and signal proc- 
essor 2008. 

[0059] So that the user-defined video, sub-picture, 
and audio portions of the multimedia title edited by the 
authoring encoder EC are reproduced, the authoring de- 
coder DC comprises a scenario selector 21 00 for select- 
ing and reproducing the corresponding scenes (titles). 
The scenario selector 2100 then outputs the selected 
titles as scenario data to the authoring decoder DC. 
[0060] The scenario selector 2100 preferably com- 
prises a keyboard, CPU, and monitor. Using the key- 
board, the user then inputs the desired scenario based 
on the content of the scenario input by the authoring en- 
coder EC. Based on the keyboard input, the CPU gen- 
erates the scenario selection data St51 specifying the 
selected scenario. The scenario selector 2100 is con- 
nected by an infrared communications device, for ex- 
ample, to the decoding system controller 2300, to which 
it inputs the scenario selection data St51 . 
[0061] Based on the scenario selection data St51 , the 
decoding system controller 2300 then generates the bit- 
stream reproduction control signal St53 controlling the 
operation of the bitstream producer 2000. 
[0062] The stream buffer 2400 has a specific buffer 
capacity used to temporarily store the reproduced bit- 
stream St61 input from the bitstream producer 2000, ex- 
tract the address information and initial synchronization 
data SCR (system clock reference) for each stream, and 
generate bitstream control data St63. The stream buffer 
2400 is also connected to the decoding system control- 
ler 2300, to which it supplies the generated bitstream 
control data St63. 

[0063] The synch ro n izer 2900 i s con n ected to th e d e- 
coding system controller 2300 from which it receives the 
system clock reference SCR contained in the synchro- 
nization control data St81 to set the internal system 
clock STC and supply the reset system clock St79 to the 
decoding system controller 2300. 
[0064] Based on this system clock St79, the decoding 
system controller 2300 also generates the stream read 
signal St65 at a specific interval and outputs the read 
signal St65 to the stream buffer 2400. 
[0065] Based on the supplied read signal St65, the 
stream buffer 2400 outputs the reproduced bitstream 
St61 at a specific interval to the system decoder 2500 
as bitstream St67. 

[0066] Based on the scenario selection data St51 ,-the 
decoding system controller 2300 generates the decod- 
ing signal St69 defining the stream Ids for the video, sub- 
picture, and audio bitstreams corresponding to the se- 
lected scenario, and outputs to the system decoder 
2500. 

[0067] Based on the instructions contained in the de- 
coding signal St69, the system decoder 2500 respec- 
tively outputs the video, sub-picture, and audio bit- 



streams Input from the stream buffer 2400 to the video 
buffer 2600, sub-picture buffer 2700, and audio buffer 
2800 as the encoded video stream St71 , encoded sub- 
picture stream St73, and encoded audio stream St75. 

5 [0068] The system decoder 2500 detects the presen- 
tation time stamp PTS and decoding time stamp DTS of 
the smallest control unit in each bitstream St67 to gen- 
erate the time information signal St77. This time infor- 
mation signal St 77 is supplied to the synchronizer 2900 

10 through the decoding system controller 2300 as the syn- 
chronization control data St81 . 
[0069] Based on this synchronization control data 
St81 , the synchronizer 2900 determines the decoding 
start timing whereby each of the bitstreams will be ar- 

15 ranged in the correct sequence after decoding, and then 
generates and inputs the video stream decoding start 
signal St89 to the video decoder 3800 based on this de- 
coding timing. The synchronizer 2900 also generates 
and supplies the sub-picture decoding start signal St91 

20 and audio stream decoding start signal St93 to the sub- 
picture decoder 3100 and audio decoder 3200, respec- 
tively. 

[0070] The video decoder 3800 generates the video 
output request signal St84 based on the video stream 

25 decoding start signal St89, and outputs to the video buff- 
er 2600. In response to the video output request signal 
St84, the video buffer 2600 outputs the video stream 
St83 to the video decoder 3800. The video decoder 
3800 thus detects the presentation time information 

30 contained in the video stream St83, and disables the vid- 
eo output request signal St84 when the length of the re- 
ceived video stream St83 is equivalent to the specified 
presentation time. A video stream equal in length to the 
specified presentation time is thus decoded by the video 

35 decoder 3800, which outputs the reproduced video sig- 
nal St1 04 to the synthesizer 3500. 
[0071] The sub -picture decoder 3100 similarly gener- 
ates the sub-picture output request signal St86 based 
on the sub-picture decoding start signal St91 , and out- 

^0 puts to the sub-picture buffer 2700. In response to the 
sub-picture output request signal St86, the sub-picture 
buffer 2700 outputs the sub-picture stream St85 to the 
sub-picture decoder 3100. Based on the presentation 
time information contained in the sub-picture stream 

« St 85, the sub-picture decoder 31 00 decodes a length of 
the sub-picture stream St85 corresponding to the spec- 
ified presentation time to reproduce and supply to the 
synthesizer 3500 the sub-picture signal St99. 
[0072] The synthesizer 3500 superimposes the video 

so signal St104 and sub-picture signal St99 to generate 
and output the multi-picture video signal St1 05 to the 
video data output terminal 3600. 
[0073] The audio decoder 3200 generates and sup- 
plies to the audio buffer 2800 the audio output request 

5 5 signal St88 based on the audio stream decoding start 
signal St93. The audio buffer 2800 thus outputs the au- 
dio stream St87 to the audio decoder 3200. The audio 
decoder 3200 decodes a length of the audio stream 
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